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ABSTRACT 

During  the  various  phases  of  the  Defence  acquisition  process,  and  in  the  early  design  phases,  many 
decisions  must  be  made  concerning  the  performance,  crew  configuration  and  cost  of  new  equipment,  e.g. 
new  naval  platforms.  In  practice  many  of  these  decisions  are  made  either  based  on  a  limited  view  on  their 
impact  or  based  on  subjective  information.  Moreover,  previous  studies  have  shown  that  the  impact  of  the 
decisions  taken  in  the  early  design  phases  is  large;  they  generally  determine  as  much  as  80%  of  the  total 
life  cycle  costs.  This  highlights  the  importance  of  decision  making  support  in  these  areas. 

In  case  of  the  acquisition  of  new  equipment,  the  various  alternatives  must  be  assessed  on  performance  and 
costs,  and  the  constitution  of  the  crew  needed  for  operating  the  equipment  has  to  be  determined.  In  case  of 
the  design  of  new  platforms  also  the  design  issues  have  to  be  addressed.  This  assessment  can  be  done  at 
various  levels  of  detail:  performance  of  the  platform  during  its  lifetime  in  various  missions,  but  also  the 
composition  and  performance  of  the  crew  during  specific  tasks. 

TNO  has  developed  two  methodologies  to  support  the  acquisition  and  design  process  of  new  equipment 
and  the  assessment  of  manning  concepts:  the  manning  concept  framework  and  the  Simulation  Based 
Performance  Assessment  ( SBPA )  methodology. 

Together  these  methodologies  constitute  a  toolbox  to  support  the  process  of  designing  a  new  platform  or 
upgrading  an  existing  platform  on  both  the  technical  and  the  human  components. 


1  INTRODUCTION 

The  operational  environment  of  the  Dutch  Ministry  of  Defence  (NL  MoD)  is  very  dynamic  and  complex. 
The  nature  of  the  missions  changes,  advanced  technology  plays  a  large  role  and  the  number  of  people  with 
which  the  missions  can  be  carried  out,  decreases.  The  NL  MoD  strives  for  an  optimal  balance  between 
operational  effectiveness  and  life  cycle  costs  of  its  materiel.  Because  of  the  sea  platform  requirements  of 
the  Royal  Netherlands  Navy  (RNLN),  the  RNLN  fulfils  the  role  of  smart  designer  and  smart  integrator 
during  the  acquisition  process.  The  RNLN  acts  as  smart  designer  of  its  platforms  and  as  smart  integrator 
of  the  Sensor,  Weapon  and  Command  (SEWACO)  systems.  The  Royal  Netherlands  Air  Force  and  Royal 
Netherlands  Army  usually  act  as  smart  buyer. 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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The  NL  MoD  uses  various  techniques  to  support  various  forms  of  design  and  re-design  processes.  For 
instance  during  the  acquisition  process  (‘Defensie  Materieel  Proces’,  DMP),  during  the  maintenance  and 
modernization  programmes,  and  during  the  implementation  of  modifications  and  for  doctrine -related 
questions,  The  use  of  these  techniques  is  currently  somewhat  fragmented  and  is  primarily  aimed  at  the 
operational  assessment  of  specific  subsystems. 

In  order  to  support  better  decision  making  during  the  various  forms  of  design  and  re-design  processes,  the 
Netherlands  Organisation  for  Applied  Scientific  Research  TNO  has  developed  both  the  manning  concept 
framework  and  the  Simulation  Based  Performance  Assessment  (SBPA)  methodology  that  can  support  an 
integral  and  systematic  approach  on  both  technical  and  manning  aspects  at  various  design  levels. 

At  platform  level  the  SBPA  methodology  enables  the  determination  of  the  overall  performance  of  a 
platform.  It  considers  the  operational  effectiveness,  survivability,  sustainability,  and  life  cycle  costs.  These 
aspects  are  compared  in  a  transparent  and  unbiased  manner.  In  this  way,  the  SBPA  methodology  enables 
to  perform  trade-offs  between  different  design  alternatives  and  thus  allows  well-informed  acquisition 
decision-making.  An  integrated  performance  assessment  is  supported  by  simulation  models,  since  they 
enable  the  evaluation  of  the  system  as  a  whole  beforehand  and  reduce  the  risks  and  costs  of  real-life 
experimentation. 

Manning  plays  an  important  role  in  the  performance  of  a  platform  and  the  design  choice  of  manning 
concept  will  have  a  large  influence  on  the  (operational)  effectiveness  of  the  platform  and  its  associated 
development  and  life  cycle  costs.  Therefore,  the  design  and  acquisition  process  of  a  platform  requires  an 
integrated  approach  in  which  the  manning  concept  should  be  aligned  with  the  technical  possibilities  to 
meet  the  overall  functional  and  performance  requirements  which  are  set  in  the  initial  design  phase. 
Furthermore,  design  alternatives  should  be  considered  in  light  of  the  pay-off  between  effectiveness  and 
associated  development  and  life  cycle  costs.  The  manning  concept  framework  is  an  integrated  and 
systematic  approach  that  utilizes  the  design  of  manning  concepts  and  supports  the  decision  making 
process  when  considering  design  alternatives  at  the  various  design  level,  including  the  relationship 
between  ship  design  and  manning  concepts. 

This  paper  presents  the  manning  concept  framework  and  the  SBPA  methodology  and  shows  how  these  are 
related  and  how  they  can  concurrently  be  used  in  a  design  process. 

The  remainder  of  this  paper  is  organised  as  follows.  Sections  2  and  3  introduce  the  manning  concept 
framework  and  the  SBPA  methodology.  Section  4  describes  how  they  are  related.  Finally,  we  present  our 
conclusions. 


2  MANNING  CONCEPT  FRAMEWORK 

This  section  describes  the  manning  concept  framework  shown  in  Figure  1 .  A  more  extensive  description 
can  be  found  in  Reference  [8].  The  framework  consists  of  four  interconnected  design  levels: 

•  strategic  ambition; 

•  functional  demands; 

•  conceptual  design; 

•  detailed  design. 
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Figure  1 :  The  manning  concept  framework. 


Each  level  views  the  platform  at  a  different  level  of  aggregation.  It  means  for  instance  that  on  the 
functional  demands  level  considerations  are  made  regarding  the  deployment  of  a  platform  and  that  at  the 
detailed  design  level  job  designs  of  a  single  crew  member  are  established  as  well  as  the  operations  room 
lay-out.  Factors  within  one  level  a  strongly  interconnected,  e.g.  at  conceptual  design  level  the  choice  of  an 
organisational  structure  influences  the  tasks  and  work  processes.  Breaking  the  design  process  down  into 
four  design  levels  is  one  design  principle  that  has  been  applied. 

The  second  design  principle  is  that  the  levels  are  interconnected.  It  means  that  the  choices  made  on  one 
level  constrain  the  design  choices  on  subsequent  levels.  Constraints  have  to  be  relaxed  on  a  higher  level 
when  they  can  not  be  matched  within  the  associated  development  and  life  cycle  costs  boundaries  on 
subsequent  levels.  It  could  mean  that  initial  ambition  levels  and  task  execution  concepts  have  to  be 
reconsidered  because  they  are  not  realistic  in  terms  of  crew  size  and  costs. 

The  third  principle  applied,  is  that  at  each  level  design  alternatives  can  be  compared  in  terms  of  their 
effectiveness  and  associated  development  and  life  cycle  costs.  Because  the  design  levels  vary  concerning 
their  level  of  aggregation  also  the  cost  and  effectiveness  indications  vary  in  precision.  At  a  high 
aggregation  level  only  relative  changes  can  be  indicated,  where  on  the  detailed  design  level  more  detailed 
and  absolute  indications  can  be  provided. 

It  is  important  to  note  that  the  manning  concept  framework  does  not  generate  design  solutions,  but  it  is 
supportive  to  the  design  process  carried  out  by  humans  and  improves  their  decision-making.  It  is 
supportive  because  it  provides  an  integrated  overview  of  the  design  process,  it  shows  the  interconnection 
between  various  design  choices  at  various  levels,  and  it  allows  that  the  rationales  behind  design  choices 
are  made  explicit  and  well  documented. 
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It  improves  decision-making  because  the  frame  work  allows  pay-off  consideration  between  investment 
and  development  costs  on  the  one  hand  and  life  cycle  costs  on  the  other  hand.  Furthermore,  by  applying 
the  framework  the  feasibility  of  what  is  wanted  (ambition  and  functionality)  at  what  price  (manning  and 
technology)  can  be  established  during  the  design  process  instead  of  only  finding  out  in  the  operational 
phase  of  a  platform.  The  framework  helps  to  decrease  the  possibility  of  operational  and  financial  setbacks. 

2.1  Strategic  ambition 

The  acquisition  of  a  new  platform  starts  with  the  strategic  ambition.  The  strategic  ambition  is  usually 
based  on  political  decisions  regarding  the  deployment  of  the  armed  forces  and  their  main  assets.  These 
political  decisions  give  direction  to  the  design  process  and  provide  the  preconditions  that  the  design  must 
meet.  The  strategic  ambition  includes  aspects  such  as: 

•  Level  of  violence  in  which  the  platform  has  to  operate. 

•  Task  performance:  does  the  platform  have  to  carry  out  its  main  tasks  simultaneously  or  one  main 
task  at  a  time?  Can  external  support  be  called  upon? 

•  Endurance  ambition:  how  long  should  the  platform  be  able  to  operate  without  external  support 
and  supply? 

•  Technology  level:  is  advanced  technology  requested  or  is  a  ‘simple’  platform  good  enough. 

•  Reliability,  Availability,  Maintainability  (RAM):  Is  a  high  level  of  RAM  required  or  not? 

•  Training:  can  be  ashore  or  on  board,  whether  or  not  supported  by  coaching. 

•  Is  the  crew  size  constrained  or  not? 

The  other  three  levels  are  the  “real”  design  levels.  In  general  we  can  characterise  the  levels  by  the  focus  of 
the  design: 

•  Functional:  functioning  of  the  platform  itself 

•  Conceptual:  functioning  of  manning  teams 

•  Detail:  functioning  of  a  single  crew  member 

2.2  Functional  demands 

The  functional  demands  describe  the  functions  the  platform  as  a  whole  should  be  able  to  execute.  They 
describe  the  missions  and  tasks  that  the  platform  has  to  carry  out,  e.g.  coast  guard,  maritime  interdiction, 
Anti  Submarine  Warfare.  The  scenarios  in  which  the  platform  is  expected  to  operate  and  to  carry  out  these 
missions  are  also  part  of  the  functional  demands  level.  For  each  of  the  tasks,  Measures  of  Effectiveness 
(MOEs)  are  defined  that  can  be  used  to  evaluate  the  design.  Minimum  requirements  to  these  MOEs  (the 
functional  demands)  can  lead  to  limiting  conditions  for  the  conceptual  and  detailed  design  levels.  The 
evaluation  at  this  level  is  most  closely  related  to  the  SBPA  methodology,  which  will  be  described  in 
Section  3. 

2.3  Conceptual  design 

At  the  conceptual  design  level,  the  functional  demands  are  translated  to  working  methods:  the  main 
structure  of  the  work  organisation  including  specific  working  processes.  Because  the  conceptual  design 
level  is  right  between  ambition  and  functional  demands  on  one  side  and  (practical)  detailed  design 
solutions  on  the  other  side,  it  can  be  seen  as  the  most  creative  process  in  which  innovative  solutions  can  be 
sought.  Aspects  of  the  manning  concepts  that  occur  at  this  level  are  the  organisation  structure,  procedures, 
team  tasks  and  processes,  allocation  of  tasks  to  crew  members,  the  technical  environment  and 
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architectures.  Examples  of  a  conceptual  design  are  an  integrated  command  bridge,  modular  teams  etc.  The 
choices  that  will  be  made  at  the  conceptual  design  level  are  the  limiting  conditions  for  the  detailed  design. 

2.4  Detailed  design 

In  the  detailed  design  phase  choices  made  at  the  higher  levels  are  tested  on  feasibility  and  desirability  by 
assigning  people  and  resources.  Organisation  structures,  working  procedures  and  tasks  as  described  at  the 
conceptual  design  level  are  further  elaborated.  For  instance:  cooperation  between  individuals  is  assessed, 
tasks  are  assigned  to  individuals,  and  the  working  environment  is  evaluated. 

Conceptual  design  and  detailed  design  are  worked  out  in  an  iterative  way  until  a  feasible  and  acceptable 
manning  and  technology  plan  is  obtained. 

2.5  Linking  functional  demands  and  conceptual  design 

The  tasks  of  the  platform  at  the  functional  demand  level  are  connected  with  the  manning  concept  aspects 
at  the  conceptual  design  level.  However,  a  direct  connection  is  difficult  to  obtain.  Therefore  seven  clusters 
have  been  identified.  These  clusters  each  have  influence  on  the  platform  tasks  at  the  functional  demand 
level.  On  the  other  hand,  the  clusters  can  be  more  easily  identified  and  linked  to  conceptual  designs,  e.g. 
because  a  cluster  corresponds  to  one  or  more  teams.  The  identified  clusters  are: 

•  Command  and  Control  (C2); 

•  Intelligence,  Surveillance  and  Reconnaissance  (ISR); 

•  Manoeuvre; 

•  Logistics  &  Sustainability; 

•  Weapon  Deployment; 

•  Force  Protection; 

•  System  management  &  Damage  Control. 

Table  1  gives  an  example  of  possible  relationship  between  tasks  and  the  functional  demand  level  and  the 
identified  clusters.  In  this  table,  the  symbols  indicates  the  strength  of  the  relationship  between  the  cluster 
in  the  corresponding  column  and  the  task  in  the  row,  where  +  and  ++  mean  a  strong  relationship,  =  an 
average  relationship  (it  has  influence,  but  not  decisive  on  the  performance  of  a  task),  and  -  and  —  indicate 
that  there  is  only  a  weak  relationship  or  no  relation  at  all. 
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Table  1 :  Example  of  the  relationship  between  tasks  and  clusters 

Weapon  Force  System 

C2  ISR  Manoeuvre  Logistics  deployment  protection  management 


Detection  =  ++  =  -  -  ... 

Interception  =  =  ++  -  =  ... 

Boarding  =  -  +  -  =  + 

Sustainability-  -  -  ++  -  =  ++ 

Survivability  =  =  =  -  +  ++  + 


2.6  Evaluating  costs  and  effectiveness 

Manning  concepts  are  evaluated  on  effectiveness  and  costs.  Especially  at  a  conceptual  and  detailed  design 
level,  quantifying  effectiveness  is  difficult.  Therefore,  a  qualitative  evaluation  is  usually  carried  out. 
During  this  evaluation,  concepts  are  compared  to  each  other  and  for  all  aspects  at  a  level,  it  is  indicated  if 
a  design  performs  (much)  better  or  worse  than  the  base  case  design.  This  is  illustrated  by  a  colour  coding 
analogous  to  the  system  of  a  traffic  light.  An  example  of  such  an  evaluation  can  be  found  in  Figure  2. 
Green  means  a  positive  effect,  orange  and  red  a  (strong)  negative  effect.  In  white  is  indicated  that  there  is 
no  difference,  a  grey  shadings  means  that  the  effect  is  unknown  and  further  investigation  is  required. 

Effectiveness  Costs 

Organisational  structure 


Span  of  control 
Responsibilities 


Procedures 

Formality 

Fit 

Actuality 

Documentation 

Completeness 

Team  characteristics 

Team  size 
Team  structure 
Team  composition 
Team  maturity 
Team  knowledge 


Architecture 

Working  place 

System  design 


Figure  2:  Qualitative  evaluation  at  conceptual  design  level. 


2.7  Role  plan 

If  a  manning  concept  is  feasible  and  cost  effective  at  all  design  levels,  this  leads  to  a  so  called  ‘role  plan’, 
which  describes  how  the  ambition  level  will  be  met,  which  missions  and  tasks  will  be  carried  out,  which 
organisation  structures  and  working  process  will  be  used,  which  people  are  carrying  out  the  tasks  and  what 
technological  support  is  available.  For  all  choices,  effectiveness  and  costs  are  given. 
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2.8  Using  the  framework 

As  indicated  already,  the  frame  work  can  be  used  for  various  design  and  evaluation  processes  ranging 
from  new  platform  design  to  the  redesign  within  platform  modernisation  programmes  and  evaluating 
existing  platforms. 

A  typical  way  to  go  through  the  model  for  new  platform  design  is  the  top-down  approach.  Starting  with 
the  strategic  ambition,  the  functional  demands  are  derived.  Based  on  these  functional  demands,  the 
conceptual  design  phase  starts  by  creating  and  evaluating  conceptual  designs  and  deciding  which 
conceptual  design  is  the  most  cost  effective.  Based  on  the  best  conceptual  design,  the  detailed  design  is 
developed.  This  approach  is  suitable  for  research  questions  with  an  open  and  innovative  character  when 
new  concepts  are  not  limited  by  details  and  budget.  The  approach  is  illustrated  in  the  left  flowchart  of 
Figure  3. 


Figure  3:  Top-down  approach  (left)  and  iterative  approach  (right). 

However  in  practice  an  unlimited  design  process  is  less  realistic.  Strategic  ambitions  may  cause  limitations 
at  various  levels  and  limits  on  crew  size  and  the  budget  for  technology  development  cause  limitations  on 
the  possible  solutions.  This  indicates  that  a  design  process  is  an  iterative  process  by  nature,  which  means 
that  the  various  phases  have  to  be  run  through  iteratively  as  well.  The  strategic  ambition  of  the  RNLN  with 
respect  to  the  platform  is  usually  fixed  when  the  design  process  starts.  Starting  with  the  design  process 
itself  is  possible  at  the  functional  demands  level,  but  also  at  the  conceptual  design  phase.  The  strategic 
ambition  (e.g.  constrained  crew  size  or  high  technology  level)  might  induce  the  need  for  new  manning 
concepts.  In  this  situation,  the  conceptual  design  phase  is  the  starting  point  of  the  design  process  (see  the 
right  side  of  Figure  3). 

The  concepts  developed  at  the  conceptual  design  level  are  not  only  evaluated  on  costs  and  effectiveness  at 
this  level,  but  also  on  the  implications  on  functional  demands  level  (i.e.  can  the  functional  demands  still  be 
met?).  A  conceptual  design  leads  to  the  consideration  of  suitable  detailed  designs.  Infeasibilities  at  one  of 
the  level  lead  to  an  adjustment  of  the  original  design  and  the  phases  have  to  be  run  through  again.  The 
detailed  design  phase  will  usually  not  be  the  final  step. 

In  this  way,  the  phases  are  run  through  in  an  iterative  way  until  a  design  has  been  found  which  matches  the 
constraints  and  strategic  and  functional  ambitions. 
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3  SIMULATION  BASED  PERFORMANCE  ASSESSMENT  (SBPA) 
METHODOLOGY 

This  section  introduces  the  SBPA  methodology  and  shows  its  relation  to  other  approaches. 

The  SBPA  methodology  allows  a  transparent,  unbiased  and  integral  performance  assessment  of  (future) 
platforms.  It  defines  an  evaluation  methodology  to  assess  the  overall  performance  of  a  platform  to  support 
the  acquisition  process.  It  is  based  on  Multi  Criteria  Analysis  (MCA)  and  simulation  techniques. 

The  US  DoD  has  developed  the  Cost  and  Operational  Effectiveness  Analysis  (COEA)  approach  [1],  The 
goal  of  COEA  is  to  perform  a  cost-benefit  analysis  to  support  acquisition  decisions.  Important  aspects  of 
the  COEA  approach  are  the  problem  definition,  determining  alternative  system  concepts,  defining  system 
requirements  and  MOEs,  defining  scenarios,  data  collection  and  model  selection,  and  estimating  costs  and 
operational  effectiveness  of  the  possible  system  concepts. 

Based  on  the  COEA,  the  Integrated  Cost  and  Operational  Effectiveness  Approach  (ICEA)  was  developed, 
in  the  NATO  Future  Reduced  Cost  Combatant  study  [2],  The  ICEA  analyses  design  alternatives  in  the 
early  design  stage  using  operational  effectiveness  and  life  cycle  cost.  It  defines  scenarios  and  missions, 
and  it  develops  a  mission  task  tree.  The  operational  effectiveness  is  calculated  using  the  Operational  Value 
Model  (OVM)  [3].  The  survivability  and  sustainability  are  calculated  separately.  ICEA  combines  the 
operational  effectiveness  and  the  survivability  and  sustainability  into  an  overall  Figure  of  Merit  (FOM)  for 
the  platform  using  the  MCA  Analytic  Hierarchy  Process  [4]. 

Both  OVM  and  ICEA  use  an  aggregation  method  to  combine  the  different  MOEs  into  operational 
effectiveness  and  an  overall  performance  of  the  platform.  These  aggregation  methods  are  common  to 
Multi  Criteria  Analysis  (MCA)  [5].  MCA  is  a  scientific  evaluation  method  for  making  rational  choices 
between  several  discrete  alternatives.  It  evaluates  and  compares  the  alternatives  to  obtain  an  aggregated 
assessment  which  results  in  a  ranking  of  the  alternatives.  Typical  for  most  MCA  processes  are  preparing 
an  evaluation  structure  (model)  and  dealing  with  the  subjective  preferences  of  one  or  more  stakeholders 
involved  and  the  processing  of  information  from  diverse  and  diffuse  nature  and  origin  (conflicting, 
quantitative  and  qualitative,  inexact  and  uncertain,  et  cetera). 

The  SBPA  methodology  leverages  these  approaches  and  uses  the  following  elements  in  particular  to 
establish  a  structured  process: 

•  the  COEA  approach 

•  mission-task  analysis  (ICEA  and  OVM) 

•  simulation  models  with  different  levels  of  aggregation  and  fidelity 

•  MCA  techniques  for  weighing  and  comparing  the  different  components 

The  SBPA  methodology  consists  of  five  phases  each  consisting  of  several  steps,  see  also  Figure  4.  We 
will  now  give  a  short  overview  of  the  five  phases.  A  detailed  description  of  the  step  by  step  plan  can  be 
found  in  [6]. 
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Figure  4:  SBPA  methodology  step  by  step  plan. 


3.1  Phase  1:  Definition  phase 

In  this  phase  the  operating  conditions  of  the  platform  are  defined:  the  expected  missions  and  tasks  that  will 
be  carried  out  by  the  platform  under  consideration  as  well  as  the  expected  scenarios  in  which  the  platform 
operates  are  described.  Since  exercises  are  an  important  part  of  the  usage  of  a  platform,  they  should  be 
taken  into  account  to  properly  determine  sustainability  and  cost. 

During  the  definition  phase  a  structured  description  of  the  assumptions  made  and  expected  scenarios  is 
produced. 
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3.2  Phase  2:  Product  effectiveness 

In  the  second  phase  the  evaluation  framework  is  developed.  An  important  part  of  the  evaluation 
framework  consists  of  determining  the  overall  effectiveness  of  the  platform,  the  so-called  product 
effectiveness.  Product  effectiveness  consists  of  three  parts:  operational  effectiveness,  survivability  and 
sustainability.  These  are  defined  as: 

•  Operational  effectiveness:  The  ability  to  perform  military,  diplomatic,  police  and  humanitarian 
missions  and  tasks. 

•  Survivability:  The  ability  to  sail  and  fight  after  suffering  an  attack  and  the  accumulation  of  battle 
damage. 

•  Sustainability:  The  ability  to  ensure  the  availability  of  propulsion,  manoeuvrability,  weapon, 
sensor  and  support  systems  to  keep  the  platform  operational. 


For  each  of  these  parts,  the  criteria  that  are  to  be  evaluated  are  placed  in  an  MCA  tree.  An  MCA  tree 
contains  both  the  criteria  on  which  a  design  will  be  assessed  and  the  relationships  between  these  criteria. 
For  the  operational  effectiveness,  this  tree  is  usually  called  a  mission  task  tree,  because  this  tree  describes 
the  relations  between  scenarios,  missions  and  tasks  and  their  relative  importance,  given  by  weights  that  are 
determined  in  general  by  expert  opinion.  For  sustainability,  criteria  such  as  reliability,  maintainability, 
required  personnel  and  spare  parts  are  commonly  used. 

For  each  criterion,  one  or  more  Measures  of  Effectiveness  (MOEs)  are  formulated  to  evaluate  the 
performance  of  the  platform  on  this  criterion.  Also,  it  is  determined  how  the  MOEs  are  translated  to  a 
common  scale,  by  using  utility  functions.  In  this  way,  MOEs  can  be  meaningfully  combined. 

This  evaluation  framework  is  setup  in  advance,  in  order  to  avoid  bias  in  the  evaluation  process.  This 
framework  enables  a  smooth  evaluation  process  for  the  various  design  variants.  Because  all  evaluation 
aspects  are  now  described,  new  design  variants  can  be  added  later  on,  following  the  same  procedure. 
Moreover,  it  enables  a  fair  comparison  of  alternatives. 

A  part  of  the  setup  of  the  evaluation  framework  is  the  decision  how  the  product  effectiveness  of  a  design  is 
determined,  i.e.  using  expert  opinion,  simulation  or  analytical  models.  Simulation  models  are  an 
invaluable  tool.  However,  careful  consideration  is  needed  regarding  the  choice  of  input  parameters  and 
variations  in  the  simulation  input  to  ensure  that  the  simulation  results  are  representative  for  the  real 
effectiveness  of  the  platform. 

3.3  Phase  3:  Life  Cycle  Costs 

The  third  phase  defines  how  the  Life  Cycle  Costs  (LCC)  are  determined  [7] .  The  LCC  of  a  new  platform 
can  be  divided  into  6  main  categories: 

•  Design 

•  Development 

•  Production 

•  Operating 

•  Support 

•  Disposal 
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Each  category  consists  of  a  number  of  subcategories  for  which  the  costs  can  be  determined  in  various 
ways  and  with  various  levels  of  detail. 

The  three  cost  estimation  methods  that  are  used  mostly  are  ‘analogy’,  ‘parametric’  and  ‘engineering’.  All 
these  methods  can  be  used  in  the  SBPA  methodology. 

The  analogy  method  compares  a  new  system  (design)  with  one  or  more  existing  systems  and  does  not 
only  take  into  account  technical  differences,  but  also  differences  in  use  and/or  maintenance  concepts. 

The  parametric  method  estimates  the  cost  based  on  various  measurable  properties  of  the  system.  It  is 
based  on  the  existence  of  a  causal  relationship  between  system  cost  and  system  properties. 

Unlike  the  parametric  method,  in  the  engineering  method  the  formulas  are  based  on  the  detailed 
consideration  of  the  system  and  its  use,  and  the  relations  between  them.  This  method  requires  detailed 
design  information. 

3.4  Phase  4:  Evaluation  of  each  alternative 

The  first  three  phases  have  defined  the  evaluation  framework  that  will  be  used  in  this  phase  to  assess  the 
performance  and  life  cycle  cost  for  each  of  the  alternatives.  In  this  way  the  alternatives  can  each  be 
assessed  in  an  identical  manner. 

This  phase  determines  the  actual  performance  for  each  of  the  alternatives.  Using  the  results  of  Phase  2,  it 
calculates  values  for  each  of  the  defined  MOEs,  and  it  aggregates  these  MOE  values  into  Figures  of  Merit 
(FOMs)  for  each  of  the  three  product  effectiveness  parts.  This  may  involve  simulation  as  well  as  using 
expert  opinions.  And,  using  the  results  of  phase  3,  it  performs  the  calculation  of  the  UCC. 

3.5  Phase  5:  Analysis  of  results  and  comparison  of  alternatives 

In  the  final  phase  the  results  are  analysed,  the  alternatives  are  compared  on  product  effectiveness  and  costs 
and  a  sensitivity  analysis  is  carried  out  to  determine  the  robustness  of  the  results. 

3.6  Use  of  the  SBPA  methodology 

When  using  the  SBPA  methodology,  it  is  important  to  pay  thorough  attention  to  each  of  the  steps.  Putting 
down  in  writing  the  steps  followed  and  choices  made,  as  well  as  the  model  input,  is  essential  for  a  proper 
evaluation  and  comparison  of  design. 

Although  it  may  be  difficult,  it  is  important  to  describe  the  operational  context  of  the  platform  adequately. 
This  includes  for  example,  the  life  cycle,  characteristics  of  the  areas  of  operation  and  of  the  behaviour 
(tactics,  level  of  aggression)  of  opponents  and  own  platforms.  Care  must  be  taken  that  the  assumptions 
that  are  made  have  a  broad  coverage  and  are  plausible  at  the  same  time. 

As  applying  the  SBPA  methodology  is  quite  time  consuming,  it  is  recommended  to  determine  timely  the 
required  level  of  detail.  For  example,  if  the  contribution  of  one  task  to  the  overall  FOM  is  very  low, 
because  of  the  low  weights  for  this  task  and  the  missions  it  belongs  to,  it  should  be  considered  to  address 
this  task  in  less  detail,  i.e.  with  less  effort. 
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The  ‘definition  phase’  specifies  the  input  for  generating  MOEs  for  the  different  tasks.  However  extra 
assumptions  have  to  be  made  when  setting  up  and  executing  the  simulations,  for  example,  many  additional 
assumptions  are  made  regarding  environmental  conditions  and  system  parameters.  These  assumptions 
must  be  recorded  and  consistency  among  them  must  be  assured. 

The  sensitivity  and  uncertainty  analysis  during  the  comparison  of  alternatives  is  a  very  important  part  of 
the  methodology.  It  provides  insight  in  the  robustness  of  the  performance  assessment. 

The  SBPA  methodology  can  be  used  in  an  iterative  way  to  find  better  platform  designs.  Starting  with  a 
limited  number  of  designs,  the  evaluation  can  be  used  to  generate  improved  designs.  These  hopefully 
improved  designs  can  be  evaluated  by  going  through  Step  4  and  5  of  the  methodology  again.  Note  that  the 
first  three  steps  are  describing  how  the  evaluation  should  be  carried  out.  In  this  iterative  way,  improved 
designs  can  be  found. 


4  INTERACTION  BETWEEN  MANNING  CONCEPT  FRAMEWORK  AND 
SBPA  METHODOLOGY 

The  design  of  a  new  ship  comprises  both  the  ship  structure  and  characteristics  and  the  manning  on  board 
the  ship.  Choices  made  regarding  the  required  capabilities  of  the  new  platform,  the  platform  design  and 
the  manning  concept  interact  with  each  other. 

The  manning  concept  framework  is  primarily  meant  for  -  as  the  name  indicates  -  the  development  and 
evaluation  of  manning  concepts.  However,  the  structure  with  various  levels  of  detail  is  also  valid  for  the 
technical  design  of  a  new  platform.  Strategic  ambitions  form  the  starting  points  of  every  design  process. 
The  SBPA  methodology  focuses  on  the  evaluation  of  the  operational  performance  and  life  cycle  costs  at 
platform  level,  i.e.  the  functional  demands  level.  Examples  of  design  decision  at  this  level  are  the 
maximum  speed  of  the  platform,  the  availability  of  a  helicopter,  the  signature  and  sensor  and  weapon 
systems.  At  conceptual  design  level  the  design  aspects  are  focussed  on  parts  of  the  ship  and  ways  to  obtain 
the  specifications  of  the  functional  demands.  An  example  is  the  concept  of  signature  reduction  measures 
that  result  in  the  signature  given  at  functional  demands  level. 

Technical  aspects  should  be  closely  related  and  should  even  be  part  of  the  manning  concepts  when  for 
instance  automation,  technical  support  and  centralisation  enables  different  ways  of  task  executions  and 
crew  configurations.  Hence,  new  manning  concepts  must  come  together  with  technological  developments 
and  innovations.  As  shown  in  Section  2,  the  manning  concept  framework  takes  technical  aspects  into 
account.  The  starting  point  of  the  manning  concept  framework,  however,  is  a  human  centred  and 
conceptual  driven  design  process.  Of  course,  redesign  may  well  start  with  a  technology  push.  But  in  cases 
like  that,  the  human  dimension  of  the  system  should  be  re-designed  and  adjusted  accordingly.  This  shows 
that  manning  concepts,  technological  developments  and  ship  design  are  closely  related.  The  operational 
performance  is  a  combination  of  technological  innovations,  i.e.  the  platform  design,  at  the  implemented 
manning  concept.  Figure  5  shows  the  relationships  in  the  ship  design  process. 

For  the  evaluation  of  the  manning  concept  framework  at  functional  demands  level  the  SBPA  methodology 
is  an  excellent  tool.  At  conceptual  and  detailed  design  level,  evaluation  often  occurs  in  a  qualitative  way 
by  comparing  alternatives  and  indicating  if  an  alternative  performs  (much)  better  or  worse.  Simulation 
models  to  obtain  quantitative  measure  are  hardly  available. 
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As  mentioned  before,  the  design  process  of  a  new  platform  is  an  iterative  process.  This  does  not  only  hold 
for  the  construction  of  a  manning  concept,  but  also  for  the  design  of  the  platform  itself.  Platform  design 
and  manning  concept  development  interact  with  each  other.  New  technical  systems  might  change  the 
working  process  and  the  number  of  crew  members  required  for  a  certain  task.  The  size  of  the  ship  and  the 
facilities  on  board  must  be  enough  to  accommodate  the  required  manning. 
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Figure  5:  Place  of  the  methodologies  in  the  ship  design  process. 


5  CONCLUSIONS 

In  this  paper  we  have  presented  the  manning  concept  framework  and  the  SBPA  methodology  developed 
by  TNO.  The  combination  of  methodology  and  framework  assist  the  design  process  in  evaluating  design 
concepts  during  the  acquisition  process  and  for  upgrades  of  existing  platforms  at  various  design  levels. 
Many  aspects  that  are  relevant  for  the  assessment  of  new  concepts  are  taken  into  account:  operational 
effectiveness,  survivability,  sustainability,  life  cycle  costs  and  the  performance  and  the  well-being  of  the 
crew  members. 

Although  both  methodology  and  framework  have  clear  added  value,  they  do  not  provide  an  easy  recipe 
that  can  be  followed  blindly.  The  construction  of  new  platform  designs  and  manning  concepts  remains 
both  a  creative  and  decision-making  process  that  is  supported  by  methodology  and  framework.  They  do 
not  replace  the  creative  process  and  do  not  deliver  ready-made  designs. 

The  manning  concept  framework  provides  an  extensive  overview  of  components  that  are  relevant  during 
the  design  or  evaluation  process,  mainly  on  manning  concepts.  The  framework  forces  the  designer  to 
consider  choices  and  their  consequences  in  such  a  way  that  infeasible  or  expensive  solutions  and 
unexpected  possibilities  should  become  clear  at  an  early  stage  of  the  process. 
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The  SBPA  methodology  is  a  suitable  methodology  for  executing  performance  assessment  of  new 
platforms  and  systems  as  well  as  during  maintenance  and  upgrade  programs.  Based  on  our  experiences  we 
believe  that  the  use  of  the  SBPA  methodology  leads  to  better  choices  and  improves  the  transparency 
during  the  acquisition  process. 

Both  methodology  and  framework  help  to  found  and  defend  choices  and  decisions  in  the  evaluation 
process  by  providing  a  structure  that  forces  the  designer  to  consider  all  relevant  aspects.  They  serve  as  a 
tool  to  secure  available  knowledge.  Furthermore  they  provide  insight  in  (operational  and  financial) 
consequences  of  design  choices. 

The  combination  of  the  manning  concept  framework  and  the  SBPA  methodology  provides  a  powerful  tool 
to  support  the  designer  of  new  platforms  during  the  evaluation  of  new  designs,  both  technically  and 
manning  based,  on  performance  and  costs. 
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